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DETAILED ACTION 

Response to Arguments 

1 . Applicant's response to the last Office Action, filed 02/07/2008, has been entered and made of 
record. 

2. Applicant has amended claims 1, 7, and 13. Claims 4-6 have been cancelled. Claims 1, 3, and 7-19 
are currently pending. 

3. Applicants arguments filed 06/23/2008 have been fully considered but they are not persuasive. 

4. As for the interview request by applicant, the undersigned Examiner can be contacted to schedule 
an interview at applicant convenience 

In response to applicant's argument that the examiner's conclusion of obviousness is based upon improper 
hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a 
reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which 
was within the level of ordinary skill at the time the claimed invention was made, and does not include 
knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re 
McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). Applicant argues that Mittal does not mention 
a user outlining a plurality of segments and there is no monition, explicitly or implicity, of a user being 
involved with the division of the image into blocks at all. Examiner disagree with applicant since Mittal 
teaches an apparatus for dividing images into blocks (paragraph [0026]) Note that "interactive user" was 
not recited in the rejected claim 1 and it is recited in the amended claim, even though it is believed that 
Mittal et al teaches the user examiner will use a third reference that will explain the user being involved 
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with the division of the image into blocks. Although the Mittal does not explicitly explains whether an 
interactive user divides or its an automatic process but Mittal teaches user inputs in paragraph [0188], 
therefore, it is understood that there is an interactive user that divided the images into block and does the 
background modeling( i.e. outline) . Examiner used a secondary reference DeMenthon et al to teach the 
spatio temporal segmentation, in Section 5 wherein the segmentation results and outlines how the space 
time segmentation can be used to extract concise representations suitable for video retrieval. Moreover, 
DeMenthon et al teaches in figure 1 the discrete features, interest points, edges and blobs are extracted by a 
user therefore, it is obvious to one skilled the art, to extract the edge of the segments as taught by 
DeMenthon in the background modeling of Mittal in order to obtain cleaner boundaries and reduce the 
noise quality of the video data. .Moreover, the use of threshold which is the simplest technique according 
Mittal et al (paragraph [0188]) would imply that the system performs the division based upon threshold 
established by the user which still reads on claim 1 since applicant does not explicitly explains on what 
base the division occurs and the image can be divided and outlined based on a threshold or any other 
technique. Applicant argues that Examiner does not address claims 14-19 which include the element of a 
set of edge sheets that represent the surface of the single semantic region. Examiner believed that the edge 
sheet is more as the outline surface and the semantic region is taught as forming a background model in a 
high-dimensional space, paragraph [0019]. Examiner will expand more in the rejection below. All 
remaining arguments are reliant on the aforementioned and addressed arguments and thus are considered to 
be wholly addressed herein. 

Examiner Notes 

5. Examiner cites particular columns and line numbers in the references as applied to 
the claims below for the convenience of the applicant. Although the specified citations 
are representative of the teachings in the art and are applied to the specific limitations 
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within the individual claim, other passages and figures may apply as well. It is 
respectfully requested that, in preparing responses, the applicant fully consider the 
references in entirety as potentially teaching all or part of the claimed invention, as well 
as the context of the passage as taught by the prior art or disclosed by the examiner 



Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed 
or described as set forth in section 1 02 of this title, if the differences between the 
subject matter sought to be patented and the prior art are such that the subject 
matter as a whole would have been obvious at the time the invention was made to 
a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was 
made. 



7. Claims 1-3,7-19 are rejected under 35 U.S.C. 103(a) as being unpatentable by Mittal et al (US 
2005/0286764) in view of DeMenthon et al (Spatio-Temporal Segmentation of Video by Hierarchical 
Mean Shift Analysis, July 2002) further in view of Mojsilovic et al ( US 7,043,474) 

As to claim 1, Mittal teaches a method for stylizing video, comprising: performing a spatio- 
temporal segmentation analysis on the video to identify three dimensional volumes (optical flow 
computation, spatio-temporal domain, paragraph [0012]) of contiguous pixels having a similar color; 

receiving an interactive user input (paragraph [0188], user input, note that examiner interprets 
that the division based upon threshold established by a user shows that there is a user interface and thus 
the division is performed into blocks with respect to a user ) identifying a group of the three dimensional 
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volumes (forming a background model in a high-dimensional space, paragraph [0019]; wherein the three 
dimensional volumes of contiguous pixels comprise segments (apparatus for dividing the image into 
blocks, paragraph [0026]), Note that the optical flow constraint equation is applied at a given point 
defined by a spatial location and time to obtain respective constraints; applying an error function to 
combine the respective constraints from each the given point within a defined region for deriving a 
characteristic function; deriving a motion estimate from the characteristic function; and comparing the 
motion estimate with a given uncertainty model so as to derive a figure of uncertainty for optical flow 
measurement data, paragraph [0022], paragraph [0078])wherein the interactive user input comprises 
outlining a plurality of segments While Mittal et al meets a number of the limitations of the claimed 
invention, as pointed out more fully above, Mittal fails to specifically teach the spatio-temporal 
segmentation having a similar color by mean shift analysis and outlining of plurality of segments . 
Specifically, DeMenthon et al. teaches the use of spatio temporal segmentation of video sequences by 
adopting a hierarchical clustering method, which operates by repeatedly applying mean shift analysis over 
increasingly high ranges and perform a 3D segmentation of the video stack where a foreground object is in 
front of a similar color background object. Moreover, DeMenthon teaches segment the spatio temporal 
pixel volume of the video stack with respect to feature component including color component, frame 
coordinates and optical flow components. DeMenthon teaches the region segmentation with respect to 
different features which include edges because the segmentation analysis of DeMenthon includes similar 
color segmentation and motion segmentation, and consistent labeling of regions over time which amounts 
to region tracking. It would have been obvious to one of ordinary skill in the art to use the spatio-temporal 
segmentation analysis with similar color in Mittal in order to provide a hierarchical segmentation of data 
thus obtaining cleaner boundaries and help in video indexing and retrieval and a better video quality. 
Mojisilovic et al teaches segmenting with respect to a user interface an input image by employing texture 
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segmentation, color segmentation and foregoing / background segmentation where the foreground / 
background segmentation includes using the texture map and the region map to determine if there is at 
least one dominant object in the image, and to form an object map. The input image, texture map, region 
map and object map are further processed to compute for each region in the region map, and for each 
object in the object map, a set of local features that may include, but need not be limited to, size, 
boundary, neighbors, boundary curvature, texture, mean color, color name, and shape properties for each 
region/object. Moreover, Mojisilovic teaches a texture map is generated by computing a set of directional 
edge maps, followed by a region growing procedure. For color segmentation it is preferred to use a mean- 
shift algorithm (see paragraph 125, figures 15A-15B). It would have been obvious to one of ordinary skill 
in the art to interact the user involve the user in the system of Mittal in view of DeMenthon in order to 
have an accurate categorization can be achieved by exploiting the important visual properties of each 
modality. Therefore, the claimed invention would have been obvious to one of ordinary skill in the art at 
the time of the invention by applicant. 

As to claim 3, Mittal teaches the method of claim 1, wherein the spatio-temporal 
segmentation analysis comprises an anisotropic kernel mean shift segmentation 
procedure (anisotropic non-linear filters can be considered, paragraph [0129]) 

As to claim 7, Mittal teaches the method of claim 6, wherein the outlining 
(morphological operation) is performed on a number of keyframes of the video, the 
number of keyframes being fewer than a total number of frames of the video (note that 
the image is divided into blocks, and for each block, PCA is performed using a certain 
number of previous frames, paragraph [0026-0028], figure 1) 
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As to claim 8, Mittal teaches the method of claim 7, wherein additional segments 
on frames of the video other than keyframes are identified by determining a relationship 
of the additional segments to the segments outlined on the keyframes, (figures 9-11; 
paragraph [0079]. 

As to claim 9 and 10, Mittal teaches relationship comprises at least a portion of 
the additional segments being enclosed by one or more of the segments outlined on the 
keyframes and at least a portion comprises at least a majority of pixels of the additional 
segments (The method is able to capture the long term dynamic characteristics of the 
scene, temporal and structural relationships between different pixels and multiple 
modalities of dynamic behavior, paragraph [0109], [0110] and see also DeMenthon ( 
frame coordinate, pages 3-4). 

As to claim 11, Mittal teaches the method of claim 1, further comprising applying 
a stylization to the single semantic region (note that a simple model keeps a single 
background image, paragraph [0046]). 

As to claim 12, Mittal teaches the method of claim 1 1, wherein the stylization 
comprises a mean shift technique (The Variable Bandwidth Mean shift and Data-Driven 
Scale Selection, paragraph [0082]) 

Claims 13 differ from claims 1 only in that claims 13 is a computer claim 
whereas, claims 1 is a method claim. Thus, claim 13 is analyzed as previously discussed 
with respect to claim 1 above. 

As to claim 14, Mojislovic et al teaches the computer-readable medium of 
claim 13, wherein the instructions further comprise deriving a set of edge sheets that 
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represent the surface of the single semantic region and associating the edge sheets 
with the semantic region (figure 8, figure 15A-15B). 

As to claims 15 and 16, Mojislovic et al teaches the computer-readable 
medium of claim 14, further comprising rendering the edge sheets as a curve between 
the semantic region and another portion of the video and wherein a thickness of the 
edge sheets is determined based on criteria associated with the single semantic region 
(image segmentation into regions of uniform color or texture, and further analysis of 
these regions, yields opposite results for the natural and man-made categories. 
Important characteristics of the man-made images are primarily straight lines, straight 
boundaries, sharp edges, and geometry. On the other hand, regions in images of 
natural scenes have rigid boundaries and random distribution of edges (column 14, 
lines 34 — column 15, lines 13, see also DeMenthon figure 9). 

As to claim 17, DeMenthon et al teaches a computer-readable medium of 
claim 16, wherein the criteria comprise a position of the edge sheet relative to an 
arclength of the edge sheet (section 2 and 5). 

As to claims 18 and 19, DeMenthon et al teaches the criteria comprise 
duration of existence and movement of the semantic region in the video (section 3, 
figure 5 and 6). 

Conclusion 

8. Applicant's amendment necessitated the new ground(s) of rejection 
presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See 
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MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 
37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the 
advisory action. In no event, however, will the statutory period for reply expire later than 
SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to NANCY BITAR whose telephone number is (571)270- 
1041. The examiner can normally be reached on Mon-Fri (7:30a.m. to 5:00pm). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jinge Wu can be reached on 571-272-7429. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO 
Customer Service Representative or access to the automated information system, call 
800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Jingge Wu/ 

Supervisory Patent Examiner, Art Unit 2624 



Nancy Bitar 
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